Implementation of the new validator disabling strategy #2226

tdimitrov · 2023-11-08T13:57:31Z

Closes #1966, #1963 and #1962.

Disabling strategy specification here. (Updated 13/02/2024)

Implements:

~~validator disabling for a whole era instead of just a session~~ (been the case even before)
no more than 1/3 of the validators in the active set are disabled
Removes:
DisableStrategy enum - now each validator committing an offence is disabled.
New era is not forced if too many validators are disabled.

Before this PR not all offenders were disabled. A decision was made based on enum DisableStrategy. Some offenders were disabled for a whole era, some just for a session, some were not disabled at all.

This PR changes the disabling behaviour. Now a validator committing an offense is disabled immediately till the end of the current era.

Some implementation notes:

OffendingValidators in pallet session keeps all offenders (this is not changed). However its type is changed from Vec<(u32, bool)> to Vec<u32>. The reason is simple - each offender is getting disabled so the bool doesn't make sense anymore.
When a validator is disabled it is first added to OffendingValidators and then to DisabledValidators. This is done in add_offending_validator from staking pallet.
In rotate_session the end_session also calls end_era when an era ends. In this case OffendingValidators are cleared (1).
Then in rotate_session DisabledValidators are cleared (2)
And finally (still in rotate_session) a call to start_session repopulates the disabled validators (3).
The reason for this complication is that session pallet knows nothing abut eras. To overcome this on each new session the disabled list is repopulated (points 2 and 3). Staking pallet knows when a new era starts so with point 1 it ensures that the offenders list is cleared.

Handles validator disabling in the backing subsystem. The PR introduces two changes: 1. Don't import statements from disabled validators. 2. Don't validate and second if the local validator is disabled. The purpose of this change is first to ignore statements from disabled validators on the node side. This is an optimisation as these statements are supposed to be cleared in the runtime too (still not implemented at the moment). Additionally if the local node is a validator which is disabled - don't process any new candidates. --------- Co-authored-by: ordian <noreply@reusable.software> Co-authored-by: ordian <write@reusable.software>

* master: Removed TODO from test-case for hard-coded delivery fee estimation (#2042) Expose collection attributes from `Inspect` trait (#1914) `polkadot-parachain-primitives` should not depend on `frame-support`. (#1897) [testnet] Align testnet system parachain runtimes using `RelayTreasuryLocation` and `SystemParachains` in the same way (#2023) Sort the benchmarks before listing them (#2026) publish pallet-root-testing (#2017) Contracts: Add benchmarks to include files (#2022) Small optimisation to `--profile dev` wasm builds (#1851) basic-authorship: Improve time recording and logging (#2010) Application Crypto and BEEFY Support for paired (ECDSA,BLS) crypto (#1815) [ci] Run check-rust-feature-propagation in pr and master (#2012) Improve features dev-ex (#1831) Remove obsolete comment. (#2008)

substrate/frame/staking/src/pallet/impls.rs

tdimitrov · 2023-11-08T14:02:52Z

And offenders list is already being cleared here:

polkadot-sdk/substrate/frame/staking/src/pallet/impls.rs

Line 512 in bbb6631

<OffendingValidators<T>>::kill();

ordian · 2023-11-08T14:44:53Z

And offenders list is already being cleared here:

polkadot-sdk/substrate/frame/staking/src/pallet/impls.rs

Line 512 in bbb6631

<OffendingValidators<T>>::kill();

But this is the state of offenders, not disabled state in session pallet. Although, it's currently cleared on session changes, so it's fine.

tdimitrov · 2023-11-08T14:48:29Z

But this is the state of offenders, not disabled state in session pallet. Although, it's currently cleared on session changes, so it's fine.

Exactly. DisabledValidators are cleared on each session. The code I linked is called in end_session just before that. I guess it should work :)

I had a concern why the validator indices are not recalculated but as far as I can see in the code, the active set is not reshuffled between sessions within an era.

…ge::ValidateFromExhaustive` now has got named fields

…n't disable if disabled validators will go beyond the byzantine threshold

The technical comments have been resolved and the PR is audited and tested

Closes #1966, #1963 and #1962. Disabling strategy specification [here](#2955). (Updated 13/02/2024) Implements: * validator disabling for a whole era instead of just a session * no more than 1/3 of the validators in the active set are disabled Removes: * `DisableStrategy` enum - now each validator committing an offence is disabled. * New era is not forced if too many validators are disabled. Before this PR not all offenders were disabled. A decision was made based on [`enum DisableStrategy`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/primitives/staking/src/offence.rs#L54). Some offenders were disabled for a whole era, some just for a session, some were not disabled at all. This PR changes the disabling behaviour. Now a validator committing an offense is disabled immediately till the end of the current era. Some implementation notes: * `OffendingValidators` in pallet session keeps all offenders (this is not changed). However its type is changed from `Vec<(u32, bool)>` to `Vec<u32>`. The reason is simple - each offender is getting disabled so the bool doesn't make sense anymore. * When a validator is disabled it is first added to `OffendingValidators` and then to `DisabledValidators`. This is done in [`add_offending_validator`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/slashing.rs#L325) from staking pallet. * In [`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623) the `end_session` also calls [`end_era`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L490) when an era ends. In this case `OffendingValidators` are cleared **(1)**. * Then in [`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623) `DisabledValidators` are cleared **(2)** * And finally (still in `rotate_session`) a call to [`start_session`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L430) repopulates the disabled validators **(3)**. * The reason for this complication is that session pallet knows nothing abut eras. To overcome this on each new session the disabled list is repopulated (points 2 and 3). Staking pallet knows when a new era starts so with point 1 it ensures that the offenders list is cleared. --------- Co-authored-by: ordian <noreply@reusable.software> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io> Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: command-bot <> Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

Closes paritytech#1966, paritytech#1963 and paritytech#1962. Disabling strategy specification [here](paritytech#2955). (Updated 13/02/2024) Implements: * validator disabling for a whole era instead of just a session * no more than 1/3 of the validators in the active set are disabled Removes: * `DisableStrategy` enum - now each validator committing an offence is disabled. * New era is not forced if too many validators are disabled. Before this PR not all offenders were disabled. A decision was made based on [`enum DisableStrategy`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/primitives/staking/src/offence.rs#L54). Some offenders were disabled for a whole era, some just for a session, some were not disabled at all. This PR changes the disabling behaviour. Now a validator committing an offense is disabled immediately till the end of the current era. Some implementation notes: * `OffendingValidators` in pallet session keeps all offenders (this is not changed). However its type is changed from `Vec<(u32, bool)>` to `Vec<u32>`. The reason is simple - each offender is getting disabled so the bool doesn't make sense anymore. * When a validator is disabled it is first added to `OffendingValidators` and then to `DisabledValidators`. This is done in [`add_offending_validator`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/slashing.rs#L325) from staking pallet. * In [`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623) the `end_session` also calls [`end_era`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L490) when an era ends. In this case `OffendingValidators` are cleared **(1)**. * Then in [`rotate_session`](https://github.com/paritytech/polkadot-sdk/blob/bdbe98297032e21a553bf191c530690b1d591405/substrate/frame/session/src/lib.rs#L623) `DisabledValidators` are cleared **(2)** * And finally (still in `rotate_session`) a call to [`start_session`](https://github.com/paritytech/polkadot-sdk/blob/bbb6631641f9adba30c0ee6f4d11023a424dd362/substrate/frame/staking/src/pallet/impls.rs#L430) repopulates the disabled validators **(3)**. * The reason for this complication is that session pallet knows nothing abut eras. To overcome this on each new session the disabled list is repopulated (points 2 and 3). Staking pallet knows when a new era starts so with point 1 it ensures that the offenders list is cleared. --------- Co-authored-by: ordian <noreply@reusable.software> Co-authored-by: ordian <write@reusable.software> Co-authored-by: Maciej <maciej.zyszkiewicz@parity.io> Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: command-bot <> Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

tdimitrov and others added 6 commits October 20, 2023 16:13

Merge branch 'master' into tsv-disabling

42607ad

Merge branch 'master' into tsv-disabling

c986fd3

Merge branch 'master' into tsv-disabling

b16c8b6

Any offender is disabled for a whole era - wip

bbb6631

tdimitrov changed the base branch from master to tsv-disabling November 8, 2023 13:57

tdimitrov commented Nov 8, 2023

View reviewed changes

substrate/frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved

Fix on_offence compilation errors

c80ae21

tdimitrov added the T8-polkadot This PR/Issue is related to/affects the Polkadot network. label Nov 9, 2023

Remove DisableStrategy

8f5d2b6

tdimitrov force-pushed the tsv-disabling branch from 9e1039e to b16c8b6 Compare November 9, 2023 12:48

tdimitrov added 2 commits November 9, 2023 14:48

Merge branch 'master' into tsv-disabling

c8adffe

Merge branch 'tsv-disabling' into tsv-disabling-for-era

2507584

tdimitrov mentioned this pull request Nov 9, 2023

Disabling - feature PR #1955

Closed

4 tasks

tdimitrov added 9 commits November 9, 2023 19:09

Fix a compilation error after merge master: `CandidateValidationMessa…

bff0240

…ge::ValidateFromExhaustive` now has got named fields

Merge branch 'tsv-disabling' into tsv-disabling-for-era

e601dee

Update a comment

95aa673

Update comments

873e072

Merge branch 'master' into tsv-disabling

02814e0

Merge branch 'tsv-disabling' into tsv-disabling-for-era

5afbfe3

Replace OffendingValidatorsThreshold with disabling_threshold. Do…

129a986

…n't disable if disabled validators will go beyond the byzantine threshold

Make tests pass

ce1539c

Update tests to cover the new disabling strategy

aee9fe3

tdimitrov mentioned this pull request Nov 15, 2023

Don't disable if disabled validators will go beyond the byzantine threshold #2333

Closed

tdimitrov linked an issue Nov 15, 2023 that may be closed by this pull request

1/3 disabling limit in substrate #1963

Closed

tdimitrov changed the title ~~Validators are disabled for a whole era on any offence~~ Implementation of the new validator disabling strategy Nov 15, 2023

Merge branch 'master' into tsv-disabling-for-era

1e90461

ordian enabled auto-merge April 26, 2024 13:26

ordian added this pull request to the merge queue Apr 26, 2024

Merged via the queue into master with commit 988e30f Apr 26, 2024
139 of 140 checks passed

ordian deleted the tsv-disabling-for-era branch April 26, 2024 14:02

This was referenced May 2, 2024

[Meta] State of Disabling #4359

Open

Rm content related to Im-Online Pallet w3f/polkadot-wiki#5825

Merged

Being offline is no longer a slashable offense w3f/polkadot-wiki#5800

Closed

bkontur pushed a commit that referenced this pull request May 15, 2024

Renamings (#2226)

cca5f10

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 16, 2024

Renamings (#2226)

e787683

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 17, 2024

Renamings (#2226)

f80da5b

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 17, 2024

Renamings (#2226)

25396bc

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 17, 2024

Renamings (#2226)

d24b371

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 20, 2024

Renamings (#2226)

057426d

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 21, 2024

Renamings (#2226)

6006298

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 22, 2024

Renamings (#2226)

45ed4dc

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 23, 2024

Renamings (#2226)

72973a6

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request May 30, 2024

Renamings (#2226)

cff071b

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request Jun 4, 2024

Renamings (#2226)

09daaad

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request Jun 5, 2024

Renamings (#2226)

ef05c84

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request Jun 7, 2024

Renamings (#2226)

5e68120

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

bkontur pushed a commit that referenced this pull request Jun 7, 2024

Renamings (#2226)

291865a

* StorageProofSize -> StorageSize * Rename benchmarks * StorageSize -> UnverifiedStorageProofParams * Fix clippy * Fix priority boost

peterwht mentioned this pull request Aug 15, 2024

chore: upgrade to 1.12.0 r0gue-io/pop-node#193

Merged

This was referenced Aug 21, 2024

Update polkadot-sdk from v1.11.0 to stable2407 moondance-labs/tanssi#659

Open

Update polkadot-sdk from v1.11.0 to stable2407 moonbeam-foundation/moonbeam#2912

Closed

rdoria1 mentioned this pull request Oct 24, 2024

Update polkadot 1.15.0 zkVerify/zkVerify#161

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of the new validator disabling strategy #2226

Implementation of the new validator disabling strategy #2226

tdimitrov commented Nov 8, 2023 •

edited by Overkillus

Loading

tdimitrov commented Nov 8, 2023

ordian commented Nov 8, 2023

tdimitrov commented Nov 8, 2023

Implementation of the new validator disabling strategy #2226

Implementation of the new validator disabling strategy #2226

Conversation

tdimitrov commented Nov 8, 2023 • edited by Overkillus Loading

tdimitrov commented Nov 8, 2023

ordian commented Nov 8, 2023

tdimitrov commented Nov 8, 2023

tdimitrov commented Nov 8, 2023 •

edited by Overkillus

Loading